Single Root I/O Virtualization Research Articles

MiddleNet: A Unified, High-Performance NFV and Middlebox Framework With eBPF and DPDK

Traditional network resident functions (e.g., firewalls, network address translation) and middleboxes (caches, load balancers) have moved from purpose-built appliances to softwarebased components. However, L2/L3 network functions (NFs) are being implemented on Network Function Virtualization (NFV) platforms that extensively exploit kernel-bypass technology. They often use DPDK for zero-copy delivery and high performance. On the other hand, L4/L7 middleboxes, which have a greater emphasis on functionality, take advantage of a full-fledged kernelbased system. L2/L3 NFs and L4/L7 middleboxes continue to be handled by distinct platforms on different nodes. This paper proposes MiddleNet that develops a unified network resident function framework that supports L2/L3 NFs and L4/L7 middleboxes. MiddleNet supports function chains that are essential in both NFV and middlebox environments. MiddleNet uses the Data Plane Development Kit (DPDK) library for zero-copy packet delivery without interrupt-based processing, to enable the ’bumpin-the-wire’ L2/L3 processing performance required of NFV. To support L4/L7 middlebox functionality, MiddleNet utilizes a consolidated, kernel-based protocol stack for processing, avoiding a dedicated protocol stack for each function. MiddleNet fully exploits the event-driven capabilities of the extended Berkeley Packet Filter (eBPF) and seamlessly integrates it with shared memory for high-performance communication in L4/L7 middlebox function chains. The overheads for MiddleNet in L4/L7 are strictly load-proportional, without needing the dedicated CPU cores of DPDK-based approaches. MiddleNet supports flow-dependent packet processing by leveraging Single Root I/O Virtualization (SR-IOV) to dynamically select the packet processing needed (Layers 2 -7). Our experimental results show that MiddleNet achieves high performance in such a unified environment.

Integrated SDN-NFV 5G Network Performance and Management-Complexity Evaluation

Digitalization is one of the factors that affects the acceleration of the application of telecommunications technologies such as 5G. The 5G technology that has been developed today does not yet meet different performance and manageability standards, particularly for data center networks as a supportive technology. Software-defined networking (SDN) and network function virtualization (NFV) are two complementary technologies that are currently used by almost all data centers in the telecommunications industry to rectify performance and manageability issues. In this study, we deliver an integrated SDN-NFV architecture to simplify network management activities in telecommunication companies. To improve network performance at the computing level, we performed a modification of a networking system at the computing level, underlying NFV devices by replacing the default virtual switch with a data plane development kit (DPDK) and single root I/O virtualization (SR-IOV). This study evaluated the proposed architecture design in terms of network performance and manageability. Based on 30 days of observation in prime time, the proposed solution increased throughput up to 200 Mbps for the server leaf and 1.6 Gbps for the border leaf compared to the legacy architecture. Meanwhile, the latency decreased to 12 ms for the server leaf and 17 ms for the border leaf. For manageability, we tested three different scenarios and achieved savings of 13 min for Scenario 1, 22 min for Scenario 2 and 9 min for Scenario 3.

Virtualizing GPU direct packet I/O on commodity Ethernet to accelerate GPU-NFV

Network functions (NFs) play an important role in the rapid and dynamic deployment of various services. Network function virtualization (NFV) on commodity servers is becoming popular; and is associated with benefits in terms of cost, elasticity, and liveness, instead of dedicated hardware middleboxes. In this recent trend, graphics processing units (GPUs) have been shown to be highly suitable for NF acceleration, owing to their high parallelism and memory bandwidth. This paper proposes a new GPU-NFV architecture called Janus to accelerate NFs in virtualized environments. Janus is an extension of a previous study conducted by the present authors and involves GPU direct packet input/output (I/O) on commodity Ethernet, to a virtualization environment. Janus uses single-root I/O virtualization (SR-IOV) to enable high-performance NIC sharing between the host and guest virtual machine running GPU direct packet I/O. Internally, the received packets for GPU-NFV are transferred to the GPU memory directly through a combination of SR-IOV and peer-to-peer DMA. The evaluation results show that Janus’ GPU-NFV performance is up to 2.3x higher than that of the state-of-the-art data-plane development kit (DPDK).

FlexVF: Adaptive network device services in a virtualized environment

Single-root I/O virtualization (SR-IOV) allows virtual machines direct access to physical network cards through so-called virtual functions (VFs), considerably reducing networking overhead compared to paravirtualized interfaces by avoiding the hypervisor’s network stack. However, the maximum number of VFs on each card is often limited compared to the number of VMs running on each host, leading to the problem of choosing which VMs to allocate these VFs. In this paper, we introduce FlexVF, a mechanism for dynamically allocating and switching between VFs and paravirtualized networking on VMs based on network activity monitoring. We show that FlexVF improves VM network performance by 75% without affecting network operation.

When I/O Interrupt Becomes System Bottleneck: Efficiency and Scalability Enhancement for SR-IOV Network Virtualization

High performance networking interface cards (NIC) have become essential networking devices in commercial cloud computing environments. Therefore, efficient and scalable I/O virtualization is one of the primary challenges on virtualized cloud computing platforms. Single Root I/O Virtualization (SR-IOV) is a network interface technology that eliminates the overhead of redundant data copies and the virtual network switches through direct I/O in order to achieve nearly natural I/O performance. However, the SR-IOV still suffers from serious problems due to the high overhead for processing excessive network interrupts as well as the unpredictable and bursty traffic load in high-speed networking connections. In this paper, the defects of SR-IOV with 10 Gigabit Ethernet networking are studied first and two major challenges are identified: excessive interrupt rate and single threaded virtual network driver. Second, two interrupt rate control optimization schemes, called coarse-grained interrupt rate (CGR) control and adaptive interrupt rate (AIR) control are proposed. The proposed control schemes can significantly reduce the overhead and enhance the SR-IOV performance compared with the traditional driver with fixed interrupt throttle rate (FIR). In addition, multi-threaded VF driver (MTVD) is proposed that allows the SR-IOV VFs to leverage multi-core resources in order to achieve high scalability. Finally, these optimizations are implemented and detailed performance evaluations are conducted. The results show that CGR and AIR can improve the throughput by 2.26x and 2.97x while saving the CPU resources by 1.23 core and 1.44 core, respectively. The MTVD can achieve 2.03x performance with additional 1.46 cores consumption for VM using the SR-IOV driver.

Efficient routing and reconfiguration in virtualized HPC environments with vSwitch‐enabled lossless networks

SummaryTo meet the demands of communication‐intensive workloads in the cloud, virtual machines (VMs) should utilize low overhead network communication paradigms. In general, such paradigms enable VMs to directly communicate with the hardware by means of a passthrough technology like Single‐Root I/O Virtualization (SR‐IOV). However, when passthrough‐based virtualization is coupled with lossless interconnection networks, live migrations introduce scalability challenges due to the substantial network reconfiguration overhead. With these challenges in mind, we proposed a virtual switch (vSwitch) SR‐IOV architecture for InfiniBand in our previous work titled “Towards the InfiniBand SR‐IOV vSwitch Architecture”. In this paper, we first suggest solutions to rectify the space‐domain scalability issues that are present in vSwitch‐enabled subnets as a result of the VMs using dedicated layer‐two addresses. Then, we discuss routing strategies for virtualized environments using vSwitches and present a routing algorithm for Fat‐Trees. We also present a reconfiguration method that minimizes imposed reconfiguration overhead on Fat‐Trees. We perform an extensive evaluation of our prototype algorithms, and as vSwitch‐enabled hardware does not yet exist, we deduce from empirical observations by emulating vSwitches with existing hardware, as well as large‐scale simulations. Our results show significant reduction in the reconfiguration times as route recalculations can be eliminated, and for certain scenarios, the number of reconfiguration subnet management packets sent to switches is reduced from several hundred thousand down to a single one without degrading the routing quality.

A Hypervisor Approach to Enable Live Migration with Passthrough SR-IOV Network Devices

Single-Root I/O Virtualization (SR-IOV) is a specification that allows a single PCI Express (PCIe) device (physical function or PF) to be used as multiple PCIe devices (virtual functions or VF). In a virtualization system, each VF can be directly assigned to a virtual machine (VM) in passthrough mode to significantly improve the network performance. However, VF passthrough mode is not compatible with live migration, which is an essential capability that enables many advanced virtualization features such as high availability and resource provisioning. To solve this problem, we design SRVM which provides hypervisor support to ensure the VF device can be correctly used by the migrated VM and the applications. SRVM is implemented in the hypervisor without modification in guest operating systems or guest VM drivers. SRVM does not increase VM downtime. It only costs limited resources (an extra CPU core only during the live migration pre-copy phase), and there is no significant runtime overhead in VM network performance.

SRVM

Single-Root I/O Virtualization (SR-IOV) is a specification that allows a single PCI Express (PCIe) device (ysical function or PF) to be used as multiple PCIe devices (virtual functions or VF). In a virtualization system, each VF can be directly assigned to a virtual machine (VM) in passthrough mode to significantly improve the network performance. However, VF passthrough mode is not compatible with live migration, which is an essential capability that enables many advanced virtualization features such as high availability and resource provisioning. To solve this problem, we design SRVM which provides hypervisor support to ensure the VF device can be correctly used by the migrated VM and the applications. SRVM is implemented in the hypervisor without modification in guest operating systems or guest VM drivers. Our experimental results show that SRVM can effectively migrate all memory state, and there is no data loss or corruption in applications after live migration. SRVM does not increase VM downtime. It only costs limited resources (an extra CPU core), and there is no significant runtime overhead in VM network performance. In fact, since the VF can continue to be used during the pre-copy ase, it offers network throughput which is 9.6 times and network latency which is 98% lower compared to other solutions that switch to para-virtualization mode during live migration.

Advances in the IBM z13 I/O function and capability

The IBM z13™ introduces a number of significant improvements in the I/O subsystem relative to the IBM zEnterprise™ EC12. This includes advancing the I/O drawer infrastructure to implement generation 3 (Gen 3) of the Peripheral Component Interconnect Express® (PCIe®) links and switches. For the first time on this platform, the PCIe ports are placed directly within the IBM z13 processor itself, reducing latency by avoiding the necessity of using hub chips that have traditionally been implemented for routing and expansion of the PCIe infrastructure. Portions of the processor chip have been reserved to implement the functionality that would previously have resided on these separate hub chips. Traditional I/O features such as Fiber Connection (FICON®), Fibre Channel Protocol (FCP), Open System Adapter (OSA), Crypto Express, and Flash Express are provided with enhanced functionality in the PCIe I/O drawer. The PCIe infrastructure has created the opportunity to integrate additional native PCI adapters. This is further exploited with the use of Single Root I/O virtualization (SR-IOV) capability on a Remote Direct Memory Access over Converged Ethernet (RoCE) Express adapter, which now provides shared exploitation of the facilities within this adapter. An adapter utilizing field programmable gate arrays has been developed to provide compression acceleration in the z13 and forms a foundation for additional varieties of acceleration in the future.

The next generation of highly reliable and secure encryption for the IBM z13

New business opportunities for cloud, analytics, mobile, and social applications depend on a secure computing infrastructure. The introduction of the IBM 4767 cryptographic coprocessor continues IBM leadership in marketplace security. The IBM 4767/Crypto Express5S is a versatile solution, offering three modes of operations on the IBM z13™ System: 1) Accelerator, 2) Common Cryptographic Architecture (CCA) Coprocessor, and 3) Enterprise PKCS #11 (public-key cryptography standard) Coprocessor. The highly programmable cryptographic coprocessor environment features a new ASIC (application-specific integrated circuit), FPGA (field-programmable gate array), and enhanced performance. The innovative internal hardware and firmware can be easily updated to achieve new security standards and requirements as well as new customer-specific functionality. The secure APIs (application programming interfaces) are designed to support standard cryptographic functions as well as specialized banking and financial functions. This is done in a way that allows the sensitive key material never to be exposed outside the physical secure boundary in a clear format. Performance benefits include the incorporation of elliptic curve cryptography (ECC) and format preserving encryption (FPE) in the hardware. For the z13, the number of logical domains has been increased from 16 to 85, allowing more system versatility. This new design also supports SRIOV (single root I/O virtualization) and the ability to customize arbitration to target SRIOV or quality of service.

Early experiences with live migration of SR-IOV enabled InfiniBand

Virtualization is the key to efficient resource utilization and elastic resource allocation in cloud computing. It enables consolidation, the on-demand provisioning of resources, and elasticity through live migration. Live migration makes it possible to optimize resource usage by moving virtual machines (VMs) between physical servers in an application transparent manner. It does, however, require a flexible, high-performance, scalable virtualized I/O architecture to reach its full potential. This is challenging to achieve with high-speed networks such as InfiniBand and remote direct memory access enhanced Ethernet, because these devices usually maintain their connection state in the network device hardware. Fortunately, the single root IO virtualization (SR-IOV) specification addresses the performance and scalability issues. With SR-IOV, each VM has direct access to a hardware assisted virtual device without the overhead introduced by emulation or para-virtualization. However, SR-IOV does not address the migration of the network device state. In this paper we present and evaluate the first available prototype implementation of live migration over SR-IOV enabled InfiniBand devices.

Workload-Aware Credit Scheduler for Improving Network I/O Performance in Virtualization Environment

Single-root I/O virtualization (SR-IOV) has become the de facto standard of network virtualization in cloud infrastructure. Owing to the high interrupt frequency and heavy cost per interrupt in high-speed network virtualization, the performance of network virtualization is closely correlated to the computing resource allocation policy in Virtual Machine Manager (VMM). Therefore, more sophisticated methods are needed to process irregularity and the high frequency of network interrupts in high-speed network virtualization environment. However, the I/O-intensive and CPU-intensive applications in virtual machines are treated in the same manner since application attributes are transparent to the scheduler in hypervisor, and this unawareness of workload makes virtual systems unable to take full advantage of high performance networks. In this paper, we discuss the SR-IOV networking solution and show by experiment that the current credit scheduler in Xen does not utilize high performance networks efficiently. Hence we propose a novel workload-aware scheduling model with two optimizations to eliminate the bottleneck caused by scheduler. In this model, guest domains are divided into I/O-intensive domains and CPU-intensive domains according to their monitored behaviour. I/O-intensive domains can obtain extra credits that CPU-intensive domains are willing to share. In addition, the total number of credits available is adjusted to accelerate the I/O responsiveness. Our experimental evaluations show that the new scheduling models improve bandwidth and reduce response time, by keeping the fairness between I/O-intensive and CPU-intensive domains. This enables virtualization infrastructure to provide cloud computing services more efficiently and predictably.

SR-IOV Based Network Interrupt-Free Virtualization with Event Based Polling

Along with the developments of networking and virtualization technologies, high speed network connections have become one of the key components in cloud computing and data-centers. Single-Root I/O Virtualization (SR-IOV) enhances the network throughput to the extent of becoming close to the line rate and achieving high scalability in the 10Gbps and higher network environments. However, the overhead of SR-IOV interrupt virtualization remains significant due to some additional trap-and-emulation overhead on the virtual interrupt controller. The higher the virtualization network connection is, the higher the interrupt frequency becomes through high bandwidth network. To mitigate this problem, we propose a smart Event-Based Polling model (sEBP), which leverages existing system events to trigger a regular packet polling such that network interrupts are eliminated from the critical I/O paths in the virtual environment. Due to the many varieties of system events, sEBP can deal with the network workload in a configurable and flexible manner. Based on a hierarchical virtualized environment, it can also be implemented either at the guest OS kernel level or at the Virtual Machine Manager (VMM) level. Since polling is much lighter than interrupt processing, sEBP significantly reduces the network processing overhead. The experimental results prove the efficiency of sEBP, which can achieve up to a 59% performance improvement and a 23% improved scalability ratio.

Increasing performance in KVM virtualization within a Tier-1 environment

This work shows the optimizations we have been investigating and implementing at the KVM (Kernel-based Virtual Machine) virtualization layer in the INFN Tier-1 at CNAF, based on more than a year of experience in running thousands of virtual machines in a production environment used by several international collaborations. These optimizations increase the adaptability of virtualization solutions to demanding applications like those run in our institute (High-Energy Physics).We will show performance differences among different filesystems (like ext3 vs ext4) when used as KVM host local storage. We will provide guidelines for solid state disks (SSD) adoption, for deployment of SR-IOV (Single Root I/O Virtualization) enabled hardware and what is the best solution to distribute and instantiate read-only virtual machine images.This work has been driven by the project called Worker Nodes on Demand Service (WNoDeS), a framework designed to offer local, grid or cloud-based access to computing and storage resources, preserving maximum compatibility with existing computing center policies and workflows.

High performance network virtualization with SR-IOV

Virtualization poses new challenges to I/O performance. The single-root I/O virtualization (SR-IOV) standard allows an I/O device to be shared by multiple Virtual Machines (VMs), without losing performance. We propose a generic virtualization architecture for SR-IOV-capable devices, which can be implemented on multiple Virtual Machine Monitors (VMMs). With the support of our architecture, the SR-IOV-capable device driver is highly portable and agnostic of the underlying VMM. Because the Virtual Function (VF) driver with SR-IOV architecture sticks to hardware and poses a challenge to VM migration, we also propose a dynamic network interface switching (DNIS) scheme to address the migration challenge. Based on our first implementation of the network device driver, we deployed several optimizations to reduce virtualization overhead. Then, we conducted comprehensive experiments to evaluate SR-IOV performance. The results show that SR-IOV can achieve a line rate throughput (9.48 Gbps) and scale network up to 60 VMs, at the cost of only 1.76% additional CPU overhead per VM, without sacrificing throughput and migration.